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Abstract 

MIT SchMUSE (pronounced “shmooz”) is a concurrent, distributed, delegation-based object-oriented 
interactive environment with persistent storage. It is designed to run in a “capricious” network envi¬ 
ronment, where servers can migrate from site to site and can regularly become unavailable. Our design 
introduces a new form of unique identifiers called globally unique tickets that provide globally unique 
time/space stamps for objects and classes without being location specific. Object location is achieved by 
a distributed hierarchical lazy lookup mechanism that we call realm resolution. We also introduce a novel 
mechanism called message deferral for enhanced reliability in the face of remote delegation. We conclude 
with a comparison to related work and a projection of future work on MIT SchMUSE. 
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1 Introduction 

MIT SchMUSE is a ScHEME-based [Clinger & Rees 91] 
[IEEE 91] Multi-User Simulation Environment based 
on a concurrent, distributed, delegation-based object- 
oriented language with a persistent object store. 
It was originally conceived as a teaching vehicle 
for the introductory programming course at MIT 
[Abelson & Sussman 85]. 

The project’s main goal was to provide a Scheme- 
based concrete introductory case-study of computer sys¬ 
tems issues such as concurrency, distributed comput¬ 
ing, persistence, recoverability, transactions, reliability, 
and delegation-based object-oriented programming sys¬ 
tem (OOPS). We did so under the guise of implementing 
a multi-user interactive adventure game, mainly because 
that is a common setting very familiar to most students. 
It was not our intention, however, to limit ourselves 
to this application domain. For example, we envision 
the SchMUSE as being a practical prototype system 
in which to implement a distributed interactive virtual 
office space or a distributed object-oriented database. 
Nonetheless, for reader accessibility, the examples in this 
report reflect a simulated world setting. Moreover, this 
report focuses exclusively on the novel features of the 
SchMUSE language design and its functionality rather 
than on distributed systems issues per se. To wit, we 
view our primary contribution as having explored several 
novel language systems issues that arise in distributed 
object delegation with regards to language implementa¬ 
tion rather than in having tackled the distributed sys¬ 
tems issues that arise merely by virtue of being a dis¬ 
tributed computing environment, such as message trans¬ 
action atomicity and concurrency coordination. 

We chose to implement our system in MIT Scheme 
[Hanson 91], a dialect of the Lisp family of program¬ 
ming languages. Our object system is modelled after the 
delegation-based OOPS style of [Adams & Rees 88], pri¬ 
marily because that is the language in which our course 
is taught. None of our results are specific to Scheme: 
we could equally as well have implemented this system 
in Smalltalk-80 [XLRG 81], CLOS [Bobrow et al. 90], 
Dylan [Apple 92] or even C++ [Stroustrup 86]. 1 

1.1 Application Context 

The laboratory setting in which we envision MIT 
SchMUSE being exercised has greatly influenced its de¬ 
sign. Specifically, our course lab is comprised of a local 
area network of 46 high-performance diskless worksta¬ 
tions (HP 720s) evenly divided across 2 high-powered 
centralized network file servers (HP 750s). Ultimately, 
however, we envision MIT SchMUSE spreading across 
the Internet as an extremely distributed global network 

1 Of course, delegation-based dialects of these lan¬ 
guages would be required. The exemplar-based dialect of 
Smalltalk [LaLonde 86] could be used. Delegation-based 
extensions can be made fairly painlessly to CLOS and Dy¬ 
lan by careful use of the Meta-Object Protocol (MOP) 
[Kiczales et al. 91], as demonstrated in [Blair & Maessen 93]. 
Finally, the feasibility of delegation in C++ was demonstrated 
in [Johnson & Zweig 91]. 


of potentially collaborating simulation sites. For exam¬ 
ple, now that MIT provides network access in the cam¬ 
pus dormitories, we would not be surprised if our stu¬ 
dents choose to leave their otherwise idle PCs active as 
SchMUSE servers, running atop publicly available DOS 
MIT Scheme [DOS Scheme], 

The classroom mode of operation for students 
SdlMUSE-ing is envisioned to be as follows: a stu¬ 
dent logs into an unused workstation and brings up a 
local SchMUSE client session. During this session, 
the student can interact (via TCP/IP sockets) with ob¬ 
jects on either of two highly available centralized file 
servers, likely creating a few new instances of standard¬ 
ized classes on the servers, subject to a modest space 
quota. 

More interestingly, the student can also extend the 
simulation system with new classes and objects on their 
client machine. These new creations can then be made 
available to other students in the lab by allowing other 
clients in the lab to establish server connections to their 
client. In this way, we blur the distinction between server 
and client since we allow clients to themselves act as 
servers to other clients. This permits very interactive 
collaboration throughout the network. A student on one 
workstation can build on the efforts of students on sev¬ 
eral other workstations, and vice versa, to build elabo¬ 
rate team experiments. 

When a student logs out of a workstation, unfortu¬ 
nately their client/server sessions must be terminated 
(to make room for the next student), but their state 
can be dumped to persistent storage. Specifically, their 
code, as well as the state of their local object instance 
database, can be dumped as files to a personal floppy 
disk or ftp’d to a remote file server. 

When the student returns to the lab (or to their dorm 
room), all their SchMUSE state can be restored from 
their last state dump and their session can continue. Of 
course, some of the other student servers they were in¬ 
teracting with may have also moved to a different work¬ 
station ( e.g so the student can sit closer to someone else 
they are collaborating with or because their reservation 
on the machine they were using had expired) or they may 
be altogether unavailable {e.g., the student who was run¬ 
ning the server session may be temporarily in transit to 
a new location or dormant during sleeping and eating 
periods). 

1.2 Outline 

The following sections deal with the difficulties of mit¬ 
igating this sort of capricious network environment of 
itinerant virtual servers, where servers can become un¬ 
available and where servers can move among worksta¬ 
tions, both with disturbing regularity. First, however, 
we establish our language context by describing the 
SchMUSE object system in section 2. In section 3, we 
proceed to describe how remote delegation is supported 
in our object system. In section 4, we show how impos¬ 
ing class structure on our delegation system enables var¬ 
ious performance enhancements for remote delegation. 
Section 5 introduces message deferral for improved reli¬ 
ability in a system where servers are itinerant. Section 6 



relates our language design to other work in the field. 
Section 7 emphasizes the unique contributions of this 
work. Finally, in section 8, we conclude by outlining 
future work in the project. 

2 The SchMUSE Object System 

The sole means of object interaction in MIT SchMUSE 
is through message passing [Agha 86]. Methods are in¬ 
voked by passing messages to objects, and instance slots 
are accessed via messages. (It will become clear below 
why we choose to distinguish object slot accesses from 
other, more compound methods). 

Unlike prototype-based delegation systems 
[Lieberman 86], our system employs class-based delega¬ 
tion. Specifically, our objects are created as instances 
of classes, where each class declares the local variables 
and methods defined on instances of the class as well 
as declaring any parent’s classes. 2 When an instance of 
a class is generated, a chain of partial objects is made 
which corresponds to the inheritance chain prescribed 
by the object’s class structure. An example is sketched 
in Figure 1. Each partial object (box), hereafter called 
a node, contains slots for the local variables as well as 
pointers to node instances of the node’s parent classes. 
Notice that a node taken with the transitive closure of 
its parent nodes constitutes an object instance. A node 
is said to delegate to its parent objects. In this way, an 
object is represented as a directed acyclic graph (DAG) 
of nodes that directly reflects the inheritance structure 
of its class definition. 

Note from Figure 1 that slot names, in addition to 
method names, can shadow one another. For example, 
persons and students both have a nickname slot. In ad¬ 
dition, as our figure suggests, an object can be the parent 
of more than one child node. In our figure, the person 
is a common parent of both a student node and an em¬ 
ployee node. 

This very general framework for sharing is what makes 
delegation-based inheritance most compelling for dis¬ 
tributed systems. Specifically, in a distributed setting, 
each object node could reside on a separate workstation. 
We call this distribution of delegate nodes across several 
sites remote delegation. Note that traditional class-based 
inheritance systems such as Smalltalk [XLRG 81], 
CLOS [Bobrow et al. 90], and C++ [Stroustrup 86] pro¬ 
vide no such distributed sharing. Instead, they flat¬ 
ten objects into one long array of slots, one per each 
unique slot name. Consequently, slot shadowing is not 
supported in these systems. Also, slot sharing among 
multiple children is not directly supported. 3 

Delegation-derived sharing allows for complex pat¬ 
terns of centralized sharing with privacy control. In our 
example, for instance, centralized sharing is illustrated 

2 We permit multiple inheritance. For simplicity, we fol¬ 
low the mechanism of left-to-riglit topological linearization 
of multiple parents described in [Snyder 86] and used as the 
default in CLOS [Bobrow et al. 90] and Dylan [Apple 92]. 

3 Some contortions involving indirection through shared 
cell data is possible. This is what we resorted to in 

[Blair & Maessen 93]. 


by the person being shared by both the student and em¬ 
ployee nodes. By centralizing the sharing, privacy can 
be enhanced; for instance, the employee child may be 
granted access to the person’s social security number 
while the student child may be denied this information. 
Moreover, by permitting multiple independent children 
we in effect provide multiple independent views of the 
same essential object. Specifically, an application that 
knows about the student view of our person may be un¬ 
aware of our employee view. Thus, a student’s advisor 
may be unaware that s/he is consulting on the side. This 
too is an important form of privacy. 

3 Remote Delegation via Globally 

Unique Tickets and Realm Resolution 

All objects in the SchMUSE are passed to methods 
via object reference. In order to support remote del¬ 
egation, these object references must embody globally 
unique identifiers [Leach et al. 82], 

3.1 Globally Unique Tickets 

In MIT SchMUSE this need for globally unique IDs for 
object references is accomplished by implementing ev¬ 
ery object reference as an object ticket. Similarly, class 
references are implemented as class tickets. Collectively, 
object and class tickets comprise what we call globally 
unique tickets. The grammar for these GUTs of the 
SchMUSE object system is shown in Figure 2. 

The first field of each ticket is not strictly necessary: 
they are introduced primarily for debugging support, al¬ 
though we will later exploit the ClassTicket within an 
ObjectTicket. A btime is the encoding of a millisecond 
real-time clock reading of when the entity was born, and 
bmachloc is the encoding of the “birth machine loca¬ 
tion”, i.e., the network address of the machine on which 
the entity was created. Together, these two data pro¬ 
vide a globally unique identifier for every object in the 
network. 4 They do not, however, provide location infor¬ 
mation if we allow SchMUSE sessions to migrate about 
the network. This is where the RealmTicket comes into 
play. 

Each server/client session maintains its own names¬ 
pace of classes and instances in the underlying Scheme 
session. A class table and object table are maintained to 
map class tickets to classes and object tickets to object 
instances. Since SchMUSE sessions are itinerant, these 
namespaces are consequently mobile. We conceptually 
divide each namespace into realms of course-grained col¬ 
lections of objects. For example, a student may be devel¬ 
oping an adventure game simulation with a Blade Run¬ 
ner Realm and a Time Bandits Realm. Each separate 
realm can be dumped to disk and transported to another 
workstation separately. 

When an object class or instance is created, it is al¬ 
ways created in some realm. This realm information is 
encoded by the RealmTicket inside the entity’s ticket. 

4 Note also that they obviate the need for distributed clock 
synchronization: so long as each workstation assures that its 
own clock progresses strictly forward, no two distinct entity 
tickets can collide in SchMUSE space-time. 
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Figure 1: An example delegation-based instance. 


ObjectTicket 

:= ClassTicket 

X 

bt ime 

X 

bmachloc x 

RealmTicket 

ClassTicket 

:= classname 

X 

bt ime 

X 

bmachloc x 

RealmTicket 

RealmTicket 

:= realmname 

X 

bt ime 

X 

bmachloc 



Figure 2: Globally Unique Tickets 


When remote sessions access a local realm and receive 
copies of its local object tickets, this realm information 
is later used to locate the object again. That is, when a 
message is sent to an object ticket, if the object ticket’s 
realm matches the realm we are in when processing the 
message, then the object must be local. The local object 
table is then used to map the object ticket to the desired 
object and the message can be processed on that local 
object. If, on the other hand, the object ticket’s realm is 
not that of a local realm, then the message is forwarded 
to the machine where the remote object’s realm resides. 

3.2 Realm Resolution 

Notice, however, that RealmTickets pointedly do not 
contain location information other than the birthplace 
of a realm. Thus, there is nothing in the ticket to de¬ 
clare where a particular realm actually resides in the net¬ 
work. For this we resort to a “Brazilian” ° distributed 
hierarchical realm resolution/location mechanism which 
we have dubbed Central Services. Central Services is 
not a single centralized entity but, rather, a distributed 
hierarchical network. Each SchMUSE session which ini¬ 
tiates server access does so by contacting some nearby 
authority and requesting access to Central Services. The 
authority designates some server within the network of 

What is, inspired by the movie Brazil. 


active SchMUSE servers to serve as realm resolution 
authority for the new session. In this way, a variety of 
load-distributed resolution hierarchies can be initiated 
by the central authority, such as N-ary trees and H-trees 
[Leighton 92]. 

The server then informs the authority and its desig¬ 
nated Central Services contact point of the realms which 
it is making available for server access. This information 
is then lazily distributed throughout the SchMUSE net¬ 
work on demand in a fashion similar to Internet names¬ 
pace service. Should this contact point become unavail¬ 
able, a new one can be requested from the nearby author¬ 
ity. The authority network, being the backbone of the 
SchMUSE, is fixed and distributed, much like Internet 
namespace service. 

Each SchMUSE session, then, maintains a cache of 
its realm lookups to map RealmTickets to real machine 
addresses. Of course, these address mappings will be¬ 
come obsolete when a remote server session terminates, 
so a time-out mechanism is implemented whereby an un¬ 
successful connection can resort to querying Central Ser¬ 
vices to see if the desired realm has moved. When a 
session is terminated, it is customary to therefore no¬ 
tify Central Services so that future inquires regarding 
the realm’s location can be rejected without resorting to 
repeated time-out mechanisms. 

















The details of this distributed hierarchical realm res¬ 
olution mechanism of the SchMUSE are still very much 
in the experimental phase so little can be said of its per¬ 
formance. Nonetheless, we are confident that this style of 
lazy location with eager cancellation will suit our labora¬ 
tory and campus environment well. We have also found 
that performance profiling and system debugging were 
greatly facilitated by separating an entity’s birth ma¬ 
chine location from its present location hint associated 
the object tickets. Specifically, the btime and bmachloc 
remain constant despite realm motion, providing a time- 
invariant UID by which to identify objects in the net¬ 
work. 

4 Using Class Information for Efficient 
Remote Delegation 

We turn now to the performance issue of making 
delegation-based inheritance efficient in the face of re¬ 
mote delegation. In short, we use static information from 
the class definitions to accelerate method dispatch and 
slot accesses. Most importantly, the static class infor¬ 
mation allows us to delegate directly to the node where 
a slot is located rather than traversing the full delega¬ 
tion chain of the object being manipulated. In the case 
of highly distributed delegation, this can have major 
performance improvements since superfluous delegation 
chain traversal amounts to superfluous network traffic 
and unnecessary indirection through numerous worksta¬ 
tions, tying up valuable resources and heightening the 
liability of access failure. In the case where the com¬ 
plete structure information about a node is not present 
on the SchMUSE session that is processing a message, 
we must be a bit more clever. 

In brief, the performance issues surrounding remote 
delegation can be divided along two lines of concern: 
first, the amount of structure information available con¬ 
cerning the object being manipulated; and second, the 
kind of message being processed by the object. These 
concerns naturally subdivide, in turn, as follows: regard¬ 
ing structure information, we must consider 1) if the full 
class information is co-resident with the instance, or 2) 
if we allow opaque remote classes, *.e.,if we can have 
a local instance whose local class information may be 
known but whose inherited class structure is remote and 
not known. As we shall see shortly, these two class poli¬ 
cies have different implications depending on whether 
the message involved is A) a slot access message, or B) a 
compound method message. 6 The following subsections 
address each of the four resulting combinations. 

4.1 Case 1A: Full class info; Slot access 

Returning to our example delegation in Figure 1, imag¬ 
ine we are on the machine where the student node re¬ 
sides and we wish to access the student’s place slot. 
Imagine that each ancestral delegate node resides on a 
separate machine. In such a configuration, if we were 
naive in accessing the place slot of the student, we would 
have to traverse the object’s delegation DAG through the 

®That is, a method whose body is a mixture of slot refer¬ 
ences and Scheme procedure calls. 


Person node to get to the MobileObject node where the 
place slot resides. If, however, we locally knew the ob¬ 
ject tickets in the full object class ancestry of the student 
and we further knew that the place slot was accessed 
as slot 0 of the student’s parent’s primary parent (en¬ 
coded as delegate <0,0>), then we could directly send to 
the MobileObject node a request for the value of slot 
0. This would entirely bypass the Person machine and 
would lighten the burden on the MobileObject machine 
by not engaging it to decode the place message into a 
slot access on slot 0. 

We implemented such a strategy as follows: at class 
creation time, we “compile” the slot access methods to 
operate in terms of “lexical addresses” into the delega¬ 
tion chain. Specifically, at method installation time, we 
statically compute, for each slot accessor method, both 
the offset into the delegation chain for the delegate which 
possesses the target slot and the index of that slot within 
the target delegate’s array of slots. We further arrange 
that each local node, upon creation, cache away a DAG 
representing its full delegation ancestry. In this way, 
whenever a slot access method is called on a local node, 
the encoded indirection into the delegation chain can 
be used to indirect into the cached delegation ancestry 
DAG. The encoded slot access can then be sent directly 
to the object ticket representing the target delegate. In 
our prototype implementation, we witnessed an order of 
magnitude in performance improvement when this en¬ 
hancement was installed: a test case that took 20 min¬ 
utes to complete without it took only 2-3 minutes with 
it. 7 

4.2 Case IB: Full class info; Compound 
method 

Compound methods can likewise be decomposed into 
more primitive operations. Specifically, every method 
ultimately decomposes into nested sequences of Scheme 
language primitives and slot accesses. Thus, if we can 
know the implementation of the AnimateObject’s hear 
method locally on the machine where the Student node 
inherits it, then hear messages sent to the Student can 
be decomposed on the local Student node’s machine into 
Scheme primitives and slot accesses. This means that 
the bulk of the work involved in processing a hear mes¬ 
sage can be assumed by the machine on which the mes¬ 
sage was initiated. This dramatically reduces the load 
on the remote machine where the AnimateObject node 
resides, enhancing its availability to other remote child 
nodes. In our prototype implementation, we witnessed, 
on average, a 4-5 fold decrease in load on the remote 
node when this feature was enabled. 

4.3 Cases 2A &: 2B: Remote class info; Slot or 
Method messages 

The natural question to follow from the preceding is, of 
course, what to do if the full class information (namely, 
delegate storage and inherited methods) is not available 
to some remote child. For example, imagine that the 

' This test case involved an obscenely complex remote del¬ 
egation DAG being pounded on during peek network usage 
hours. 



local slot structure and local methods for a Student are 
known at the student’s node but that the class informa¬ 
tion for a Person is not known. This may be a reasonable 
situation if, say, the Person class implementor has de¬ 
cided not to export the implementation of persons. 8 In 
this case, any message that we can decode locally using 
only information known about local students can still 
be done with some efficiency, yet any inherited message 
would be an unknown. 

In such a case, we re-package the message and forward 
it to the machine where the remote class resides, us¬ 
ing the RealmTicket in the ClassTicket of the object’s 
parent to determine its residence. 9 In this re-packaged 
message we are also careful to include the object ticket 
of the Student object for which the message was origi¬ 
nally intended. It is then the responsibility of the remote 
class to ultimately discover the method for this mes¬ 
sage. This may, in turn, involve further re-packaging to 
other remote classes. Nonetheless, assuming an applica¬ 
ble method is discovered, the invocation of this method 
on the intended object ticket will naturally result in the 
appropriate decomposition of the message into Scheme 
primitives and slot accesses. Thus, the price for keeping 
a class implementation private is the cost of performing 
all non-exported methods at the class site. Notice that 
this suggests that some methods may be exported and 
some not, depending on their depth within the inher¬ 
itance structure. Notice, further, that in the extreme 
case where no classes are exported, the remote class re¬ 
packaging strategy will ultimately result in a full traver¬ 
sal of the delegation structure. This ultimately degrades 
into the naive delegation strategy outlined at the begin¬ 
ning of case 1A. 

If the unknown re-packaged message is found to cor¬ 
respond to a simple slot access method, it is then de¬ 
sirable to request the intended object ticket to reveal 
its full ancestry so that the slot access optimization can 
proceed by using this ancestry DAG to avoid travers¬ 
ing the delegation chain, analogous to case 1A above. 10 
This is very helpful in the case where some intermediate 
ancestor is only partially opaque (like Person) but some 
deeper ancestor atop which it is built is exported (like 
MobileObject). 

If, on the other hand, the message is discovered to be a 

s For example, this class may be still under construction 
or one of its methods may use proprietary code or code under 
federal export control. 

9 This, of course, is complicated by multiple inheritance. 
In such a situation, each remote parent class would have to 
be attempted, in turn, until one of them is able to process 
the message, after which the deferral target can be cached to 
avoid repeated exhaustive searches. 

10 The reason this is only “analogous” to case 1A is that a 
trick is involved. The trick is to notice that in re-packaging 
to some delegate class, the offset into the delegation chain 
that that class’s methods will embody will not include the 
delegation through the ancestry of the intended object up 
to that remote class. This is easily handled by the remote 
class by first walking through the delegation ancestry of the 
intended object to get to the delegate ancestor corresponding 
to the remote class. Once there, it can proceed by applying 
the slot access method to that object ticket. 


compound method, then, similar to case IB, the method 
can be decomposed into more primitive operations and 
processing proceeds as sketched above. Notice that it is 
not until we tackle the message at the remote class’s host 
that we discover whether the message is a slot access or 
a compound method. 

4.4 Summary of Efficient Remote Delegation 

In closing on this issue of efficient remote delegation in 
the face of full/remote classes a few points should be 
emphasized. 

First of all, it should be noted that we intend to re¬ 
quire every local object to at least have local class infor¬ 
mation on the machine where it resides. Specifically, we 
consider it undesirable to have a local Student object 
instance, say, on a machine that does not know about 
the Student class. Such a local object would be use¬ 
less in that every message to it would ultimately have 
to be forwarded somewhere else only to have the most 
primitive slot accesses actually performed on the object 
locally. 

Second, we do not have a problem with several inde¬ 
pendent implementations of the “same” class ( e.g two 
distinct kinds of Student) since each class is stamped 
by when/where its implementation was born. Specif¬ 
ically, when a class implementation is exported, the 
ClassTicket for the class at the exporting site is em¬ 
bedded within the exported class code. Thus, distinct 
re-implementations of a class ( i.e., version updates) will 
have different ClassTickets since their timestamps (at 
least) will differ. Similarly, exporting a class to a ma¬ 
chine that then extends or otherwise modifies the class 
will again be reflected in a new distinct ticket for the 
resulting new class. To our knowledge, no other system 
has adopted such a clean approach. The use of class ver¬ 
sion numbers comes close, but requires global synchro¬ 
nization on the issuance of these version numbers. Our 
technique requires no such synchronization. We find the 
globally unique ticket mechanism to be quite elegant in 
this regard. 

Finally, it is worth re-iterating that at any point in 
the class hierarchy one might choose to export or not 
export an ancestral class implementation. Thus, hierar¬ 
chical privacy and opaque encapsulation are supported in 
the SchMUSE. This mechanism, in effect, supports ab¬ 
stract types in a distributed object-oriented framework. 
The burden, of course, is that the site of the implemen¬ 
tation is charged with processing the messages to such 
objects. We have not explored the ramifications of this 
policy in detail, but we consider it to be intriguing that 
to keep an implementation aspect private or proprietary 
one must pay the overhead of servicing requests for these 
secretive aspects oneself or arrange to have them serviced 
only by trusted authorized or licensed servers. 

5 Message Deferral for Reliable 
Remote Delegation 

We now move from the efficiency issues surrounding re¬ 
mote delegation to the reliability issues. Specifically, a 
distributed system is reliable if its performance and avail- 



ability are not affected by machines crashing. We have 
already seen how delegating directly to nodes that con¬ 
tain slots allows us to jump over intermediate delegate 
nodes. This certainly enhances reliability in the case 
where the jumped node may not have been up. 

Otherwise, if some delegate is unavailable in our 
system due, for example, to the server that supports 
that delegate not being currently “plugged into the 
SchMUSE” then we can just avoid any slot accesses 
to slots local to that delegate and hence avoid thrash¬ 
ing the Central Services in search of a realm that is not 
present and wasting time with time-outs on messages 
that cannot succeed. 

5.1 The Problem 

Beyond this, however, is a desire for more graceful degra¬ 
dation in behavior. That is, failing to process a message 
because of the inaccessibility of the node which holds 
some object slot (or some method, in the remote class 
case) seems severe. In many cases, it may be that the 
method on some node is merely a fanciful specializa¬ 
tion of some inherited method. For example, the say 
method on Student may just print something silly (like 
“Yo”) then proceed to invoke the parent Person say 
method. In such a situation, it might be desirable to 
permit the student behavior to gracefully degrade into 
normal person behavior when the student method is un¬ 
available. Thus, were we to build a GradStudent class 
that delegates to Student but choose not to export the 
implementation of the Student say method to the grad 
student, 11 then attempts to make a grad student node 
say something would be fraught with peril if the parent 
Student node were crashed. Even when a grad student 
forgets for a moment how to behave like a student, it 
would be nice if they could at least still temporarily act 
like a normal person in the meantime rather than going 
totally catatonic. 

Similarly, the node at which some slot resides may 
become unavailable. If that slot shadows some other slot 
with the same name but later in the delegation chain, 
it may sometimes be desirable to permit slot reads of 
the unavailable slot to be deferred as slot reads of the 
shadowed slot. Returning to Figure 1, for example, if 
the Student nickname slot becomes unavailable, it may 
be acceptable to read the shadowed Person nickname 
slot instead. 

We have attempted to deal with this desirability of 
graceful inheritance degradation by a mechanism we call 
message deferral. For example, we would say that when 
the student class is not available, it may be safe to defer 
the say message to the shadowed person class. Some 
methods may be safe to defer while others may not be. 
For example, when a MobileObject moves, it merely 
changes its place slot and tells the place object to up¬ 
date its inventory to include the new object (and tell 
the old place to release the object). For Persons, how¬ 
ever, we must not only move the person object but also 
move each of the objects being carried in the person’s 

11 For example, like when GradStudent behavioral norms 
are regulated by a different set of guidelines than typical 
Student norms, dude. 


possessions slot. Thus, were the person class not avail¬ 
able to some student, it would not be safe to defer move 
messages to the underlying MobileObject class upon 
which the person class was built. To do so would mean 
to lose your wallet along with all your other possessions! 

Similarly, we may have an Employee say method that 
is specialized to say “Sir” then go on to call the Person 
say method. We might then build a Manager class that 
delegates to Employee. Were we to then ask a say mes¬ 
sage of a manager whose employee node is unavailable, 
it may be undesirable, due to corporate protocol, to de¬ 
fer to the less formal person say method. Similarly, it 
might be embarrassing to defer to the pedestrian Person 
nickname slot when an attempt is made to access the 
Employee nickname slot. These illustrate a subtle com¬ 
plication in the deferral mechanism: the safety of a de¬ 
ferral is not just a property of the message or slot read 
being deferred, it is also a property of the path by which 
the deferral takes place. In our GradStudent/Student 
say the deferral to Person was acceptable, but not so for 
the Manager/Employee deferral. Thus, when attempting 
to defer a message, some record of the deferral path must 
be passed along with the deferral attempt. Notice also 
that more than one node along the delegation chain may 
be unavailable at one time. Thus, an arbitrary depth of 
deferral may become necessary. 

Finally, this deferral mechanism is further compli¬ 
cated by multiple inheritance. Consider, for example, 
some message having methods along both the primary 
parent delegation chain and the secondary parent del¬ 
egation chain. If one chain is unavailable, perhaps the 
second should be attempted. In such a situation, the 
safety and propriety of deferral may involve shadowing 
of methods not via child shadowing but via neighbor¬ 
ing parent shadowing. Viewing the child/parent paths 
as vertically directed and multiple parents of a node as 
horizontally splayed, we recognize that the child/parent 
shadowing is a matter of vertical shadowing while multi¬ 
ple parents that handle the same method could be said to 
engage in horizontal shadowing. This distinction will be¬ 
come necessary in understanding how we implement safe 
deferral. In a scenario where an employee is a student 
intern (and thus inherits from both a normal employee 
node and a student node), this issue can be critical. 

5.2 The Solution 

How then do we implement this selective deferral mech¬ 
anism? Presently, each time a new class is created that 
delegates to a parent class, the parent class is notified of 
the names of all the local messages that the child class 
handles. In this way, if the parent class has never before 
been delegated to by a child of the new child class, the 
parent class can detect which of its methods are shad¬ 
owed by the child. This same list of the child class’s local 
messages is then forwarded to the parent class’s parents’ 
classes, and so on down the class delegation chain, so 
that deeply shadowed messages can be annotated. Note 
that at each point where a shadowing is detected, we 
store in the shadowed class an association between the 
shadowed message and the child classes which shadow it. 
If we choose to declare a particular message to be safe to 



defer to some specific shadowed parent, we can specify 
that in the class definition by stating precisely, for each 
local method, which parent class(es) that method’s mes¬ 
sage can safely be deferred to. In such cases, the child’s 
message shadowing of that message is not recorded at 
the parent class. 

Note also that this shadow information is propagated 
aggressively at the earliest possible moment, namely at 
class creation time. This is because if we tried to imple¬ 
ment a lazy on-demand strategy, by the time we actually 
need to attempt a message deferral it may well be too 
late to attempt to propagate the shadowing information. 
Since this shadowing information is our means for judg¬ 
ing if a particular deferral is safe, failing to propagate 
this vital information could be our undoing. 

To see how this class shadowing information is used 
to detect deferral safety, consider the following scenario. 
An object receives a message and decomposes it into a 
slot access on some ancestor node. If this ancestor is 
unavailable, it may be desirable to attempt to defer to 
some other ancestor with a slot of the same name. We 
therefore consult the method dispatch table for an ances¬ 
tor whose class indicates that it can handle the message 
which failed, then forward the message on to the ances¬ 
tor, telling it that this is a deferral and telling it which 
ancestor(s) we have deferred across. If the deferred an¬ 
cestor is likewise unavailable, we proceed through the 
ancestry with deeper and deeper deferral attempts. Sim¬ 
ilarly, if the attempt to discover if an ancestor’s class 
can handle this message involves querying a remote class 
which is likewise unavailable then we treat that ances¬ 
tor as unavailable. If we run out of deferral candidates, 
then we finally give up and return a failure signal. On 
the other hand, if we do find some ancestor available, 
then, at the point where we succeed, the receiving an¬ 
cestor processes the message by first consulting its class’s 
association of shadowed messages and the child classes 
which shadow them. If the present message is discovered 
to be shadowed by an object class of one of the ances¬ 
tors we have just deferred across, then the deferral is 
unsafe and we send an error to that effect. Otherwise, 
we proceed with the message as usual. 

This strategy correctly detects vertical shadowing, as 
defined above. Unfortunately, it does not handle hor¬ 
izontal shadowing. For that, we need additional class 
information. Specifically, each class, upon creation, 
queries each of its parent classes to discover the full 
structure of messages that each ancestral class handles. 12 
Now, when a class is created that delegates with multi¬ 
ple inheritance, we merely arrange that the list of child 
messages that the parents are notified of includes those 
messages handled by neighboring parents. For example, 
in our Person class, the AnimateObject class would be 
notified of the local messages of Person as well as all 
messages of MobileObject (since we have left-to-right 
precedence in our multiple inheritance). When we then 
attempt a deferral that forces us to try the secondary 

12 Even with opaque remote classes, the ancestral mes¬ 
sages are revealed since this does not expose the implemen¬ 
tation. It only exposes some small structural property of the 
implementation. 


parent AnimateObject delegation chain, we include in 
our trace of the deferrals we have already attempted all 
the ancestors tried along the primary parent delegation 
chain. In this way, the horizontal shadowing information 
is given to the delegates that may be asked to process 
a deferred message, and the path which the deferral fol¬ 
lowed includes the horizontally neighboring ancestors of 
the target deferral ancestor. 

In closing, note that we have expressed this book¬ 
keeping of vertical and horizontal shadowings in terms 
of messages sent among classes. We did so because, in 
general, communication with remote classes may require 
it. In the case where a local class must communicate 
information to another local class, the message send is 
somewhat fanciful. Indeed, with some compiler exten¬ 
sions, block compiling a selection of classes could effect 
the same result directly in the internal class represen¬ 
tations. We have not explored such compiler extensions 
since we wished to make our SchMUSE implementation 
based solely on portable MIT Scheme language features. 
Compiler extensions generally do not port quite so easily. 

6 Related Work 

As mentioned at the outset, our delegation style ob¬ 
ject system was modelled after [Adams & Rees 88]. Our 
advocacy of this style’s sharing properties and sup¬ 
port of multiple shared views comes from Clovers 
[Stein & Zdonik 89], although our introduction of classes 
into the delegation style is not traditional in the sense 
that it is not prototype-based delegation [Lieberman 86]. 

Of the distributed object systems we have examined, 
nearly all have opted to extend a traditional class-based 
inheritance system to a distributed object system only to 
afterward comment that a delegation-based object sys¬ 
tem would have been more desirable, as was anticipated 
by [Otten & Hagen 90]. Most of them cite the extra flex¬ 
ibility provided by delegation systems as the motivating 
factor. [Bennett 90] even goes to the extent of identify¬ 
ing remote classes as the primary point of tension, argu¬ 
ing that delegation systems, by normally storing meth¬ 
ods along with the local slots inside objects, naturally 
satisfy this key flexibility concern. Their careful exami¬ 
nation of the design options surrounding remote classes 
was very helpful in justifying to ourselves our constraint 
that local object nodes must have local class information 
about the node’s class. Unlike them, however, we opted 
to promote classes to the status of objects themselves, 
complete with their own kind of class reference (namely, 
ClassTickets). We find our solution to this problem 
of remote classes, therefore, to be fairly elegant with¬ 
out requiring delegation-based object instances to store 
their methods directly in their representations. Rather, 
we use class information to establish a method dispatch 
table separate from the object slots array. 

Our use of globally unique tickets was inspired by 
[Leach et al. 82], It differs from the apparently com¬ 
mon use of object “proxies” for remote object references 
[Decouchant 86]. Proxies are objects that accept mes¬ 
sages and forward them to a remote object. We instead 
make our message send procedure detect if the object 



being queried is local or remote (which is easily deter¬ 
mined by inspecting the object ticket’s realm ticket) and 
remotely forward the message if it is remote. It seems 
to us that the use of proxies is an apology for having 
an awkward object reference mechanism or for having 
a system which demands every target of a message be 
an “object” in some built-in language specific sense. In 
fact, [McCullough 87] even goes so far as to then pro¬ 
vide global UIDs for their proxies, along with a table to 
map from proxies to UIDs to test object identity. This 
seems like a further apology for having used proxies at 
all. Since we were not retro-fitting our remote object 
references to an existing local object language system, 
we were not compelled to make such apologies. 

Some 

distributed object systems [Nascimento & Dollimore 92] 
[Feeley & Levy 92] have adopted version numbers for 
dealing with object and class references. A similar com¬ 
mon approach is to adopt a centralized authority to co¬ 
ordinate the issuance of sequential object ID numbers. 
Any such strategies must globally synchronize their num¬ 
bering to some extent to avoid collision. Our globally 
unique identifiers require no such synchronization. This 
makes our system scalable. To wit, our strategy could 
support a SchMUSE server at every single Internet 
address. Moreover, although we shall permit “pointer 
snapping” of RealmTickets within object/class tickets 
when those entities migrate among realms, the remain¬ 
der of the globally unique ID remains stable, as an im¬ 
mutable birthtime/birthplace of each object. This has 
greatly enhanced debugging and profiling of our system. 

Our realm resolution is currently loosely modelled af¬ 
ter Internet namespace service. We intend to investigate 
alternative distributed resolution strategies, such as the 
“clearinghouse” approach of [Oppen & Dalai 83]. Ad¬ 
mittedly, given the small size of our prototype lab net¬ 
work (48 workstations; 2 file servers), we have not been 
motivated to explore fanciful variations. Now that MIT 
has completed its installation of Internet drops within 
campus dorm rooms, the motivation to explore creative 
options is expected to grow and we anticipate a veritable 
army of participants avid to explore those options. 

Finally, our blurring of the distinction between client 
and server in the SchMUSE is similar to that found in 
Flamingo [Anderson 86]. Their system, however, is a 
cooperative mailer system; ours is an interactive simula¬ 
tion environment, so the types of client/server interac¬ 
tions are characteristically different. 

7 Contributions and Conclusions 

Our novel use of globally unique tickets as object ref¬ 
erences has proven to be elegant, versatile, low-cost, 
and scalable without requiring network-wide clock syn¬ 
chronization. Further, our separation of object identi¬ 
fication from object location, by means of embedding 
RealmTickets within object and class tickets, has proven 
quite useful. It allows course-grained object migration 
(by realm) without requiring forwarding addressing to 
disrupt the object database. This was achieved by us¬ 
ing a separate realm table to map the realm tickets to 


actual machine locations. This mapping has been proto¬ 
typed as a “Brazilian” distributed hierarchical lazy map¬ 
ping strategy modelled after Internet namespace servers. 
Again, this will scale at least as well as the Internet has 
scaled. 

We believe that our most significant contribution is 
our incorporation of class information into a delegation- 
based inheritance system, yielding what we call a class- 
based delegation system. We know of no other distributed 
object system that has wed these traditionally rivaled 
approaches of delegation and class inheritance. 

Moreover, we found that class-based delegation facili¬ 
tates several novel issues arising from remote delegation 
which have heretofore been unaddressed by the object- 
oriented community. For example, we are unaware 
of previous work in accelerating delegation through 
compile-time class analysis in a setting where classes 
themselves may be remote (and hence, compile-time 
opaque). In addition, we have also introduced a novel 
notion of message deferral which permits graceful degra¬ 
dation of object behavior and system availability as re¬ 
mote sites go down while paying heed to the safety and 
propriety of certain deferral. 

8 Future Work 

There are many directions for further development of 
MIT SchMUSE. Roughly, these include object migra¬ 
tion, enhanced concurrency, access control, and commu¬ 
nication security. 

Foremost on our agenda is investigating mechanisms 
for allowing objects to migrate from one realm to an¬ 
other. We will likely follow the fine work of the Emer¬ 
ald Project [Jul et al. 88] [Jul 88], including their for¬ 
warding address strategy [Fowler 86] [Fowler 85] (for 
“snapping” RealmTickets in stale ObjectTickets to 
point to the new realm where the object has moved) 
as well as their distributed garbage collection algo¬ 
rithm [Vestal 87] (to GC stale forwarding addresses). Of 
course, distributed GC is valuable for automatic stor¬ 
age reclamation in general anyway so that space quo¬ 
tas with manual object de-allocation do not have to be 
imposed on users. Although the above citations give 
the impression that each of these extensions is a solved 
problem, actually implementing them should prove a re¬ 
spectable engineering task. More recent advances in dis¬ 
tributed GC should prove helpful [Detlefs 90] [VAT 92] 
[Maheshwari 93]. 

Also, we would like to support a higher degree of con¬ 
currency in the SchMUSE network by providing a true 
transaction-based mechanism for our message passing. 
(Currently, our messages are not even atomic in that par¬ 
tially completed messages which fail do not back out). 
We will likely pursue a design based on nested trans¬ 
actions [Moss 81], possibly with some degree of opti¬ 
mistic concurrency among concurrent transactions. This 
should prove a fairly challenging task. 

Next, the present prototype of the SchMUSE pro¬ 
vides a fairly exotic locking strategy for object access 
control. This is documented in [Cohen 93]. At present, 
however, it provides only fine- and coarse-grained lock¬ 
ing (namely, instance locking per message and instance 



locking per instance creator). We would like to extend it 
to provide medium-grained locking as well (that is, lock¬ 
ing per instance). This is a straightforward extension. 

Our initial implementation uses [Adams & Rees 88] 
style method delegation. We intend to look into a 
more DYLAN-like generic function method specialization 
mechanism for overriding default class methods. 

Finally, it would also be interesting to support net¬ 
work privacy by encrypting messages and their results as 
they are shipped between SchMUSE sites. This should 
prove a fairly painless extension. 
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